\section{Related Work}
\label{sec:related-work}

Model-based Testing advantages have been acknowledged by different
works~\cite{DiasNeto:2007:SMT:1353673.1353681}. Most of the MBT researches have
focused on new automated tools~\cite{Pretschner:2005:OEM:1062455.1062529} or case 
studies~\cite{Dalal99model-basedtesting,
5386905, Bouquet:2005:RTA:1082983.1083282, Grieskamp:2011:MQA:1952602.1952605}.
However, as discussed by Neto et
al.~\cite{DiasNeto:2007:SMT:1353673.1353681}, few works have focused on
empirical evaluation of MBT and traditional approaches~\cite{Pretschner:2005:OEM:1062455.1062529}.

% Primeiro estudo em larga escala - complexidade
One of the first MBT studies on a large scale systems was
conducted by Dalal and Karunanithi~\cite{Dalal99model-basedtesting}. 
They compared MBT with manual testing strategies for
four large scale systems. Their work and results considered modeling
difficulties, testing coverage, and also test case implementation. However, the
complexity and difficulties of each system could neither be measured nor
correlated with the effort of applying MBT strategies. Our work
consider such factors using metrics for the system's use cases
complexity and participant's experience. Therefore, we empirically observed how
these factors influence other measured variables as the time spent and the quantity of
found defects.

% Estudo de caso da IBM - n�o emp�rico
IBM reported significant cost reductions in system testing using the MBT 
generator called GOTCHA-TCBeans~\cite{5386905}. They compared manually developed tests with MBT
automatically generated. Their conclusions are
derived mostly from gathered experience during modeling activities, but
according to the discussed case studies we could not reason the acknowledged gained percentages. 
In our study, we infer our conclusions from an experiment, in which
we defined metrics according to the literature, controlling different factors
and analyzing possible threats to its validity in order to guarantee the
soundness of the conclusions. 

Microsoft reported the usage of MBT tools and methodologies in order to
assure high quality protocol documentation.
Their work describes a methodology called Protocol Documentation Quality
Assurance Process (PQAP) that uses MBT as one of its
cornerstones~\cite{Grieskamp:2011:MQA:1952602.1952605}.
Despite being a case study, they statistically compared test suites developed
using MBT with remaining test suites for a sample of requirements tested in one
testing site.
Their evaluation has similarities with our work, correlating the effort of
applying a MBT technique and requirements documentation properties.
While their comparison uses distinct test suites for MBT and traditional 
tests ours uses the same suite for both approaches.

% Estudo emp�rico ind�stria automotiva - diferen�a nos fatores estudados
Pretschner and colleagues~\cite{Pretschner:2005:OEM:1062455.1062529} reported a
network MBT comparison in the automotive industry. Their purpose was to compare
automatically generated test suites with hand-crafted ones regarding system
testing and to the best of our knowledge this work is the only empirical study
measuring the fault detection for both approaches.
Despite their conclusions are  very similar to ours on the fault detection
overview (which states that the same amount of defects were detected by manual
and model-based tests), the conducted experiments differ in significant ways.
First, our study also focus on the time spent by
each approach. This is an important contribution considering that testing time
directly influences projects deadlines and budget. Secondly, we considered more
factors, such as the tested use cases complexity and also the participants'
experience. Finally, we explicit categorized the type of detected defects
and their severity,  proportionally testing them among the different 
techniques. 
Therefore, we believe that our work considers more factors that need to be 
acknowledged on the comparison of manual and MBT testing approaches.


% TODO: falar que testes foram todos automatizados e nenhum considerou testes de
% aceitação




% Despite most traditional works comparing model-based testing with other
% approaches, we also identified empirical works in specific niches. For
% the sake of completeness, we discuss them briefly. 
% Takeshi and Simao~\cite{6200152} presented an empirical comparison of test case
% generation methods for finite state machines. Their main purpose was to compare
% different finite state machines techniques in order to evaluate their test suite
% generations considering factors such as the number of states, inputs, outputs,
% and completness degree. 
% Korel and Koutsogiannakis~\cite{4976373} empirically evaluated model-based
% testing and code-based prioritization techniques regarding their early fault
% detection effectiveness, with the objective to evaluate test prioritization
% heuristics. From their experiment they concluded that model-based test
% prioritization may significantly improve early fault detection, but they are
% sensitive to correct/incorrect information provided by the testers/developers.

